Modèles neuronaux pour la modélisation statistique de la langue

Les modèles de langage ont pour but de caractériser et d'évaluer la qualité des énoncés en langue naturelle. Leur rôle est fondamentale dans de nombreux cadres d'application comme la reconnaissance automatique de la parole, la traduction automatique, l'extraction et la recherche d'information. La modélisation actuellement état de l'art est la modélisation "historique" dite n-gramme associée à des techniques de lissage. Ce type de modèle prédit un mot uniquement en fonction des n-1 mots précédents. Pourtant, cette approche est loin d'être satisfaisante puisque chaque mot est traité comme un symbole discret qui n'a pas de relation avec les autres. Ainsi les spécificités du langage ne sont pas prises en compte explicitement et les propriétés morphologiques, sémantiques et syntaxiques des mots sont ignorées. De plus, à cause du caractère éparse des langues naturelles, l'ordre est limité à n=4 ou 5. Sa construction repose sur le dénombrement de successions de mots, effectué sur des données d'entrainement. Ce sont donc uniquement les textes d'apprentissage qui conditionnent la pertinence de la modélisation n-gramme, par leur quantité (plusieurs milliards de mots sont utilisés) et leur représentativité du contenu en terme de thématique, époque ou de genre. L'usage des modèles neuronaux ont récemment ouvert de nombreuses perspectives. Le principe de projection des mots dans un espace de représentation continu permet d'exploiter la notion de similarité entre les mots: les mots du contexte sont projetés dans un espace continu et l'estimation de la probabilité du mot suivant exploite alors la similarité entre ces vecteurs. Cette représentation continue confère aux modèles neuronaux une meilleure capacité de généralisation et leur utilisation a donné lieu à des améliorations significative en reconnaissance automatique de la parole et en traduction automatique. Pourtant, l'apprentissage et l'inférence des modèles de langue neuronaux à grand vocabulaire restent très couteux. Ainsi par le passé, les modèles neuronaux ont été utilisés soit pour des tâches avec peu de données d'apprentissage, soit avec un vocabulaire de mots à prédire limités en taille. La première contribution de cette thèse est donc de proposer une solution qui s appuie sur la structuration de la couche de sortie sous forme d un arbre de classification pour résoudre ce problème de complexité. Le modèle se nomme Structure OUtput Layer (SOUL) et allie une architecture neuronale avec les modèles de classes. Dans le cadre de la reconnaissance automatique de la parole et de la traduction automatique, ce nouveau type de modèle a permis d'obtenir des améliorations significatives des performances pour des systèmes à grande échelle et à état l'art. La deuxième contribution de cette thèse est d'analyser les représentations continues induites et de comparer ces modèles avec d'autres architectures comme les modèles récurrents. Enfin, la troisième contribution est d'explorer la capacité de la structure SOUL à modéliser le processus de traduction. Les résultats obtenus montrent que les modèles continus comme SOUL ouvrent des perspectives importantes de recherche en traduction automatique.The purpose of language models is in general to capture and to model regularities of language, thereby capturing morphological, syntactical and distributional properties of word sequences in a given language. They play an important role in many successful applications of Natural Language Processing, such as Automatic Speech Recognition, Machine Translation and Information Extraction. The most successful approaches to date are based on n-gram assumption and the adjustment of statistics from the training data by applying smoothing and back-off techniques, notably Kneser-Ney technique, introduced twenty years ago. In this way, language models predict a word based on its n-1 previous words. In spite of their prevalence, conventional n-gram based language models still suffer from several limitations that could be intuitively overcome by consulting human expert knowledge. One critical limitation is that, ignoring all linguistic properties, they treat each word as one discrete symbol with no relation with the others. Another point is that, even with a huge amount of data, the data sparsity issue always has an important impact, so the optimal value of n in the n-gram assumption is often 4 or 5 which is insufficient in practice. This kind of model is constructed based on the count of n-grams in training data. Therefore, the pertinence of these models is conditioned only on the characteristics of the training text (its quantity, its representation of the content in terms of theme, date). Recently, one of the most successful attempts that tries to directly learn word similarities is to use distributed word representations in language modeling, where distributionally words, which have semantic and syntactic similarities, are expected to be represented as neighbors in a continuous space. These representations and the associated objective function (the likelihood of the training data) are jointly learned using a multi-layer neural network architecture. In this way, word similarities are learned automatically. This approach has shown significant and consistent improvements when applied to automatic speech recognition and statistical machine translation tasks. A major difficulty with the continuous space neural network based approach remains the computational burden, which does not scale well to the massive corpora that are nowadays available. For this reason, the first contribution of this dissertation is the definition of a neural architecture based on a tree representation of the output vocabulary, namely Structured OUtput Layer (SOUL), which makes them well suited for large scale frameworks. The SOUL model combines the neural network approach with the class-based approach. It achieves significant improvements on both state-of-the-art large scale automatic speech recognition and statistical machine translations tasks. The second contribution is to provide several insightful analyses on their performances, their pros and cons, their induced word space representation. Finally, the third contribution is the successful adoption of the continuous space neural network into a machine translation framework. New translation models are proposed and reported to achieve significant improvements over state-of-the-art baseline systems.PARIS11-SCD-Bib. électronique (914719901) / SudocSudocFranceF

Analyzing Effects of Institutional Quality on Banking Stability: Evidence from Asean Countries

Purpose: The aim of this study is to examine the effect of institutional quality on bank stability using bank-level data from 2010 to 2020.   Theoretical framework: The study considers institutions from the perspective of governance institutions. Accordingly, the concept of government institutions is related to the country's organizational foundation in terms of governance, implying institutional quality.   Design/methodology/approach: The study uses GMM method and also choose the Zscore as the primary variable for bank stability.   Findings: The results show that institutional quality increases the stability of banks. Moreover, with the threshold model, the results show that countries with institutional quality above the threshold will increase the stability of banks. In addition, macroeconomic and banking characteristics variables such as total assets, income diversification, quality of control, inflation, and GDP growth rate have a high significance in the model.   Research, Practical & Social implications: The study shows The study's empirical results have specific policy implications for the Government in implementing policies related to institutional quality to improve bank stability.   Originality/value:   there are not many researches done to investigate institutional quality to improve bank stability. Moreover, from economic crisis, the matter of banking stability is among main concerns of many researches. Second, previous researches just focus on the aspect of corruption and ignore other aspects or other factors. That’s why authors conduct this research

A Murnaghan-Nakayama rule for Grothendieck polynomials of Grassmannian type

We consider the Grothendieck polynomials appearing in the K-theory of Grassmannians, which are analogs of Schur polynomials. This paper aims to establish a version of the Murnaghan-Nakayama rule for Grothendieck polynomials of the Grassmannian type. This rule allows us to express the product of a Grothendieck polynomial with a power sum symmetric polynomial into a linear combination of other Grothendieck polynomials.Comment: 10 pages, 7 figure

Super Bound States in the Continuum on Photonic Flatbands: Concept, Experimental Realization, and Optical Trapping Demonstration

In this work, we theoretically propose and experimentally demonstrate the formation of a super bound state in a continuum (BIC) on a photonic crystal flat band. This unique state simultaneously exhibits an enhanced quality factor and near-zero group velocity across an extended region of the Brillouin zone. It is achieved at the topological transition when a symmetry-protected BIC pinned at $k=0$ merges with two Friedrich-Wintgen quasi-BICs, which arise from destructive interference between lossy photonic modes of opposite symmetries. As a proof-of-concept, we employ the super flat BIC to demonstrate three-dimensional optical trapping of individual particles. Our findings present a novel approach to engineering both the real and imaginary components of photonic states on a subwavelength scale for innovative optoelectronic devices

RMDM: A Multilabel Fakenews Dataset for Vietnamese Evidence Verification

In this study, we present a novel and challenging multilabel Vietnamese dataset (RMDM) designed to assess the performance of large language models (LLMs), in verifying electronic information related to legal contexts, focusing on fake news as potential input for electronic evidence. The RMDM dataset comprises four labels: real, mis, dis, and mal, representing real information, misinformation, disinformation, and mal-information, respectively. By including these diverse labels, RMDM captures the complexities of differing fake news categories and offers insights into the abilities of different language models to handle various types of information that could be part of electronic evidence. The dataset consists of a total of 1,556 samples, with 389 samples for each label. Preliminary tests on the dataset using GPT-based and BERT-based models reveal variations in the models' performance across different labels, indicating that the dataset effectively challenges the ability of various language models to verify the authenticity of such information. Our findings suggest that verifying electronic information related to legal contexts, including fake news, remains a difficult problem for language models, warranting further attention from the research community to advance toward more reliable AI models for potential legal applications.Comment: ISAILD@KSE 202

De novo ChIP-seq analysis

Methods for the analysis of chromatin immunoprecipitation sequencing (ChIP-seq) data start by aligning the short reads to a reference genome. While often successful, they are not appropriate for cases where a reference genome is not available. Here we develop methods for de novo analysis of ChIP-seq data. Our methods combine de novo assembly with statistical tests enabling motif discovery without the use of a reference genome. We validate the performance of our method using human and mouse data. Analysis of fly data indicates that our method outperforms alignment based methods that utilize closely related species

Chemical composition and antibacterial activities of the ethanol extracts from the leaves and tubers of Amorphophallus pusillus

Amorphophallus pusillus is a rare species which is found only in Binh Chau-Phuoc Buu Nature Reserve. In this study, we determined 7 flavonoid compounds in tuber and leaf of A. pusillus, including of vitexin, orientin, vitexin 2?-O-glucoside, cyanidin 3-O-rutinoside, pelargonidin 3-O-glucoside, schaftoside, and peonidin 3-O-rutinoside via liquid chromatography-mass spectrometry (LC-MS). Furthermore, we have proved the antibacterial activities of ethanol extracts obtained from A. pusillus leaves and tubers in the first time. The data revealed that ethanol extracts could inhibit the growth of 6 tested microorganisms, such as Bacillus cereus, Escherichia coli, Pseudomonas aeruginosa, Salmonella enteritidis, Salmonella typhimurium, and Staphylococcus aureus. These data suggested the potential application of ethanol extracts isolated from this species as natural antimicrobial agents for treatment of infection caused by bacteria, especially in dermatologic and enteric infections

ANTECEDENTS OF TOURISTS’ LOYALTY: THE ROLE AND INFLUENCE OF TOURISM PRODUCTS, DESTINATION IMAGE IN HOIAN WORLD CULTURAL HERITAGE SITE

The study’s aim is to examine the antecedents of destination loyalty, and considers the role and influence of tourism products and destination image to international tourisms’ loyalty in case of HoiAn World Cultural Heritage Site. The study suggested an integrated approach to understand tourists’ loyalty model and investigate the empirical evidence about the relationship among tourism products, destination image, risk perception, tourist experience, destination satisfaction and tourists’ loyalty. This study also mentions important questions concerning how tourism products, destination image, tourist experience, risk perception, and tourists’ satisfaction affect tourists’ loyalty. Smart PLS3 is used to estimate and test the relationships in the research model. Mediation analysis and importance performance matrix analysis are also used to consider clearly the relationship between the constructs of research model. The study’s results indicate that tourism products, destination image, tourism experience, risk perception, and satisfaction are antecedents of international tourists’ loyalty in Hoi An World Cultural Heritage Site. And in them, tourism products affect significantly positively to destination imagine and satisfaction, beside destination image and satisfaction hold the role of mediator in this relationship. Implementing IPMA to identify the predecessors that have relatively high importance for loyalty but also a relatively low performance. The results pointed out that the constructs as satisfaction, tourism product, risk perception and image have a high importance for the tourist loyalty. The study added the antecedent of tourism products and risk perception to the model and could enrich the literature, pointing to be possibility of a destination loyalty model that can be applied to various contexts, especially after COVID- 19 pandemic. The study also discussed theoretical and managerial implications for marketing tourism

Relaxed Softmax for learning from Positive and Unlabeled data

In recent years, the softmax model and its fast approximations have become the de-facto loss functions for deep neural networks when dealing with multi-class prediction. This loss has been extended to language modeling and recommendation, two fields that fall into the framework of learning from Positive and Unlabeled data. In this paper, we stress the different drawbacks of the current family of softmax losses and sampling schemes when applied in a Positive and Unlabeled learning setup. We propose both a Relaxed Softmax loss (RS) and a new negative sampling scheme based on Boltzmann formulation. We show that the new training objective is better suited for the tasks of density estimation, item similarity and next-event prediction by driving uplifts in performance on textual and recommendation datasets against classical softmax.Comment: 9 pages, 5 figures, 2 tables, published at RecSys 201